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Abstract 



We consider impulse control problems in finite horizon for diffusions with decision 
lag and execution delay. The new feature is that our general framework deals with 
the important case when several consecutive orders may be decided before the effective 
execution of the first one. This is motivated by financial applications in the trading 
of illiquid assets such as hedge funds. We show that the value functions for such 
control problems satisfy a suitable version of dynamic programming principle in finite 
dimension, which takes into account the past dependence of state process through 
the pending orders. The corresponding Bellman partial differential equations (PDE) 
system is derived, and exhibit some peculiarities on the coupled equations, domains and 
boundary conditions. We prove a unique characterization of the value functions to this 
nonstandard PDE system by means of viscosity solutions. We then provide an algorithm 
to find the value functions and the optimal control. This easily implementablc algorithm 
involves backward and forward iterations on the domains and the value functions, which 
appear in turn as original arguments in the proofs for the boundary conditions and 
uniqueness results. 
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1 Introduction 



In this paper, we consider a general impulse control problem in finite horizon of a diffusion 
process X, with intervention lag and execution delay. This means that we may intervene 
on the diffusion system at any times Tj separated at least by some fixed positive lag h, 
by giving some impulse based on the information at Tj. However, the execution of the 
impulse decided at T{ is carried out with delay mh, m > 1, i.e. it is implemented at 
time Tj + mh, moving the system from Xr T . +mh ^- to T{X^ Ti+mh - ) - , The objective is to 
maximize over impulse controls (Tj,£j)j the expected total profit on finite horizon T, of the 
form 



Such formulations appear naturally in decision-making problems in economics and finance. 
In many situations, firms or investors face regulatory delays (delivery lag), which may be 
significant, and thus need to be taken into account when management strategies are decided 
in an uncertain environment. Problems where firm's investment are subject to delivery 
lag can be found in the real options literature, for example in [2] and pQ. In financial 
market context, execution delay is related to liquidity risk (see e.g. [H]), and occurs with 
transaction, which requires heavy preparatory work as for hedge funds. Indeed, hedge 
funds frequently hold illiquid assets, and need some time to find a counterpart to buy 
or sell them. Furthermore, this notice period gives the hedge fund manager a reasonable 
investement horizon. 

From a mathematical viewpoint, it is well-known that impulse control problems without 
delay, i.e. m = 0, lead to variational partial differential equations (PDE), see e.g. the books 
[5] and [TT]. Impulse control problems in the presence of delay were studied in [H] for m 
= 1, that is when no more than one pending order is allowed at any time. In this case, 
it is shown that the delay problem may be transformed into a no-delay impulse control 
problem. The paper [5] also considers the case m = 1, but when the value of the impulse 
is chosen at the time of execution, and on infinite horizon, and these two conditions are 
crucial in the proposed probabilistic resolution. We mention also the works [3] and recently 
|12| . which study impulse problems in infinite horizon with arbitrary number of pending 
orders, but under restrictive assumptions on the controlled state process, like (geometric) 
Levy process for X and (multiplicative) additive intervention operator V. In this case, 
the problem is reduced to a finite-dimensional one where the value functions with pending 
orders are directly related to the value function without order. 

The main contribution of this paper is to provide a theory of impulse control problems 
with delay on finite horizon in a fairly general diffusion framework that deals with the im- 
portant case in applications when the number of pending orders is finite, but not restricted 
to one, i.e. m > 1. Our chief goal is to obtain a unique tractable PDE characterization 
of the value functions for such problems. As usual in stochastic control problems, the first 
step is the derivation of a dynamic programming principle (DPP). We show a suitable 
version of DPP, which takes into account the past dependence of the controlled diffusion 
via the finite number of pending orders. The corresponding Bellman PDE system reveals 



E / f(X t )dt + g(X T )+ ]T c(X {Ti+mh) -,^) . 




Ti+mh<T 
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some nonstandard features both on the form of the differential operators and their domains, 
and on the boundary conditions. Following the modern approach to stochastic control, we 
prove that the value functions are viscosity solutions to this Bellman PDE system, and 
we also state comparison principles, which allows to obtain a unique PDE characterization. 
From this PDE representation, we provide an easily implemented algorithm to compute the 
value functions, and so as byproducts the optimal impulse control. This algorithm involves 
forward and backward iterations on the value functions and on the domains, and appear 
actually as original arguments in the proofs for the boundary conditions and comparison 
principles. 

The rest of the paper is organized as follows. In Section 2, we formulate the control 
problem and introduce the associated value functions. Section 3 deals with the dynamic 
programming principle in this general framework. We then state in Section 4 the unique 
PDE viscosity characterization for the value functions. In Section 5, we provide an algo- 
rithm for computing the value functions and the optimal impulse control. Finally, Section 
6 is devoted to the proofs of results in this paper. 



2 Problem formulation 

2.1 The control problem 

Let (f2,.F, P) be a complete probability space equipped with a filtration F = {Tt)t>Q satis- 
fying the usual conditions, and W = (Wt)t>o a standard n-dimensional Brownian motion. 

An impulse control is a double sequence a = (7$, where (r^) is an increasing 

sequence of F-stopping times, and £j are J- Ti -measurable random variables valued in E. We 
require that rj+i — r, > h a.s., where h > is a fixed time lag between two decision times, 
and we assume that E, the set of impulse values, is a compact subset of R Q . We denote by 
A this set of impulse controls. 

In absence of impulse executions, the system valued in R d evolves according to : 

dX s = b(X s )ds + a(X s )dW s , (2.1) 

where b : R d R d and a -» R dxn are Borel functions on R d , satisfying usual Lipschitz 
conditions. The interventions are decided at times r, with impulse values £j based on 
the information at these dates, however they are executed with delay at times T{ + mh, 
moving the system from X( T . +mh y to X {n+mh) = T(X [jt+mh) - Here T is a mapping 
from R d x E into R d , and we assume that T is continuous, and satisfies the linear growth 
condition : 

|r(x,e)| 

su p -Mzr < °°- ( 2 - 2 ) 

(x,e)£M d xE 1 ~r l x l 

Given an impulse control a = (ji, £i)j>i € A, and an initial condition Xq € R d , the controlled 
process X a is then defined as the solution to the s.d.e. : 

X s = X + [ b(X u )du+ [ a(X u )dW u + V {r(X {n+mh ^,^)-X {Ti+mh) -)(2.3) 

J ° J ° r i+ mh<s 
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We now fix a finite horizon T < oo, and in order to avoid trivialities, we assume T — mh 
> 0. Using standard arguments based on Burkholder-Davis-Gundy's inequality, GronwalFs 
lemma and (|2.2p . we easily check that 

E[sup|Jd] < oo. (2.4) 

s<T 

Given an impulse control a = (Tj,£j)j>i £ A, we consider the total profit at horizon T, 
defined by : 

1%) = f f(X«)ds + g(XZ)+ Yl < X {n+mh)-^ 

J ° n+mh<T 

and we assume that the running profit function /, the terminal profit function g, and the 
executed cost function c are continuous, and satisfy the linear growth condition : 

sup \f(-)\ + \^)\ + \c( X ,e)\ < ^ (25) 

(x,e)&m d xE 1 + \ x \ 

This ensures with (|2.4p that 11(a) is integrable, and we can define the control problem : 

V = sup E [n(a)]. (2.6) 

Financial example 

Consider a financial market consisting of a money market account yielding a constant 
interest rate r, and a risky asset (stock) of price process (St)t governed by : 

dS t = p(S t )dt + j(St)dW t . 

We denote by Yj the number of shares in the stock, and by Zt the amount of money (cash 
holdings) held by the investor at time t. We assume that the investor can only trade 
discretely, and her orders are executed with delay. This is modelled through an impulse 
control a = (Tj,£j)j>i € A, where Tj are the decision times, and are the numbers of stock 
purchased if > or selled if £j < decided at Tj, but executed at times Tj + mh. The 
dynamics of Y is then given by 

Y t = Y + Y 

Ti+mh<t 

which means that discrete trading AYj := Yt — Y t - = £j occur at times s = Ti + mh, i > 1. 
In absence of trading, the cash holdings Z grows deterministically at rate r : dZ t = rZ t dt. 
When a discrete trading AYj occurs, this results in a variation of cash holdings by AZt ■= 
Zt — Z t - = —(AYt)St, from the self-financing condition. In other words, the dynamics of 
Z is given by 



Z + rZ u du - Y ^i- S r, 

Ti+mh<t 
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The wealth process is equal to L(St, Yt, Zt) = Zt + YtSt- This financial example corresponds 
to the general model (JOJ with X = (S, Y, Z), b = (J3 r)' , a = (7 0), and 



T(s,y,z,e) 



( 

V 



es 



Fix now some contingent claim characterized by its payoff at time T : H(St) for some 
measurable function H. The two following hedging and valuation criteria are very popular 
in finance, and may be embedded in our general framework : 

• Shortfall risk hedging. The investor is looking for a trading strategy that minimizes the 
shortfall risk of the P&L between her contingent claim and her terminal wealth, 



inf E 



H(S T )-L(S T ,Y T ,Z T ) 



• Utility indifference price. Given an utility function U for the investor, an initial capital 
z in cash, zero in stock, and k > units of contingent claims, define the expected utility 
under optimal trading 



V (z,k) 



sup E 



U(L(S T , Y T , Z T ) - kH(S t )) 



The utility indifference ask price ir a (K, z) is the price at which the investor is indifferent 
(in the sense that her expected utility is unchanged under optimal trading) between paying 
nothing and not having the claim, and receiving 7r a (ft, z) now to deliver k units of claim at 
time T. It is then defined as the solution to 



Vq(z + 7r a («,z),re) 



V (z,0). 



2.2 Value functions 

In order to provide an analytic characterization of the control problem (|2.6|) . we need as 
usual to extend the definition of this control problem to general initial conditions. However, 
in contrast with classical control problems without execution delay, the diffusion process 
solution to (|2.3p is not Markovian. Actually, given an impulse control, we see that the 
state of the system is not only defined by its current state value at time t but also by the 
pending orders, that is the orders not yet executed, i.e. decided between time t — mh and 
t. Notice that the number of pending orders is less or equal to m. Let us then introduce 
the following definitions and notations. For any t € [0, T], k = 0, . . . , m, we denote by 

Pt(k) = [p= (U,ei)i<i< k G ([0,T] x E) k : t { - U-i > K % = 2, . . . , k, 

t — mh < ti < t, i = 1, . . . , 

the set of k pending orders not yet executed before time t, with the convention that P*(0) 
= 0. For any p = (ti, ej)i<j<fc E Pt(k), t 6 [0, T], k = 0, . . . ,m, we denote 

At, p = \a = (t»,£»)»>i 6 A : {n,£i) = (U, ei), i = l,...,k and T k+1 >t\, 
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the set of admissible impulse controls with pending orders p before time t. 

For any (i, x) G [0, T] x R d , p G P t (fe), k = 0, . . . , m, and a G we denote by X t ' x ' p ' Q 
the solution to (|2.3p for t < s < T, with initial data Xt = x, and pending orders p, i.e. 



/S PS 
b(X u )du+ a(X u )dW u + Yl { T ( x (T t +mh)-,&) -X (n+mh) -). 
Jt t<n+mh<s 

Using standard arguments based on Burkholder-Davis-Gundy's inequality, Gronwall's lemma 
and (|2.2p . we easily check that 

't,x,p,a\21 < C(l + \x\ 2 ), 



E[ sup \X\ 

t<s<T 



(2.7) 



for some positive constant C depending only on b, a, T and T. We then consider the 
following performance criterion : 



J k (t,x,p, a) 



E 



t<Ti+mh<T 



for (i,sc) G [0,T] x M d , p G P t (k), k = 0, . . . , m, q = (Ti,£i)i G ^4j )P , and the corresponding 
value functions : 

v k (t,x,p) = sup J k (t,x,p,a), k = 0,...,m,(t,x,p)eV k , 

aeAt, P 

where T> k is the definition domain of v k : 

V k = {(t,x,p) :(t,x)€[0,T}xR d , peP t {k)}. 

For k = 0, -Pt(O) = 0, and we write by convention vo(t,x) = vo(t,x,$), Vq = [0, T] x M, d so 
that the original control problem in (|2.6p is given by Vq = Vq(0,Xq). Notice from (|2.5p and 
(I2.7p that the functions v k satisfy the linear growth condition on V k : 

\v k (t,x,p)\ 



sup 

(t,x, P )eT> h 



1 + \x\ 



< oo, k = 0, . . . , m. 



(2.J 



3 Dynamic programming 

In this section, we state the dynamic programming relation on the value functions of our 
control problem with delay execution. For any t G [0,T], a = (Ti,£i)j>i G A, we denote : 



i(t,a) = inf{i > 1 : Tj > t - mh} — 1 G NU{oo}, 
k(t,a) = cardji > 1 :t — mh<Ti<tj G {0, ...,m}, 
p(i, a) = (T i+t ( t|a ),^ i+t ( t)Q ,))i< i < fc ( t)Qt ) G P t (k(t,a)). 



(3.1) 
(3.2) 
(3.3) 



Theorem 3.1 TTie value functions satisfy the dynamic programming principle : for all k 
= 0, . . . ,m, (t,x,p) G V k , 



v k (t,x,p) 



sup E 



+ ^(e, a )(6 , ,^e :C,P ' a '^( ,9 ' a )) > 



(3.4) 
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where 9 is any stopping time valued in [t,T], possibly depending on a in (|3.4p . This means 
(i) for all a G At tP , for all 9 stopping time valued in [t,T], 



v k (t,x,p) > E / f(X^)ds+ Yl < X tZZh)->& 

Jt t<Ti+mh<0 

+ v k (e, a )(8,X t fj ' x ' p ' a ,p(9,a)) . 



(3.5) 



(ii) for all e > 0, there exists a G At tP such that for all 9 stopping time valued in [t,T], 
v k (t,x,p)-e < E[[ 6 f{X?™> )da+ <X^ h) _,^ 

1 t<Ti+mh<e 



+ v k{e , a )(6,X g ' ' p ' ,p(9,a)) 



(3.6) 



We now give an explicit consequence of the above dynamic programming that will be 
useful in the derivation of the corresponding analytic characterization. We introduce some 
additional notations. For all t G [0, T], we denote by X t the set of pairs (r, £) where r is a 
stopping time, t < t a.s., and £ is a .Tv- measurable random variable valued in E. For any p 
= (U, £i)i<i<k € Pt(k), we denote p- = (ti, ei)2<i< k with the convention that p_ = when 
k = 1. 

When no impulse control is applied to the system, we denote by xt' x '° the solution to 
(12. ip with initial data X t = x, and by £ the associated infinitesimal generator : 

Cip = b(x).D x <p + -tr (aa'(x)Dl<p). 

For k G {1, . . . , m}, we partition the set Pt(k) into Pf(fc) = Pf(k) U Pf{k) where 

P t \k) = [p=(t i ,e i ) 1 < l < k eP t {k) :t k >t-h} 

P?(k) = {p={ti,e i ) 1 < i < k e P t (k) :t k <t-hY 

We easily see from the lag constraint on the pending orders that Pf(k) = iff = m, and 
so Pt(m) = P/(m). 

Corollary 3.1 Let (t,x) G [0,T) x R d . 

(1) For k G {1, . . . , m}, and p = (U, ei)i<i< k G Pt(k) such that t\ + mh < T, we have for 
any stopping time 9 valued in [t, (t k + h) A (ti + m/i)) ; 



v k (t,x,p) 



E 



/(X*^ + ^M'*'>) 



(3.7) 



For k G {0, . . . , m — 1}, and p = (ij, ej)i<j<fc G Pf{k) such that t\ + m/i < T, wi/t the 
convention that Pf(k) = and t\ + m/i = T when k = 0, we have for any stopping time 9 
valued in [t, (ti + mh) A (t + h)) : 



v k (t,x,p) 



sup E 

(r,C)6Ji 



/(X^°)d S + ^(^X^' u ,p)l e< 



i,x,0 



(3. 



+ ^ +1 (0,Xf'°,pU(r,e))l r <, 
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Interpretation and remarks 

(1) Pf(k) represents the set of k pending orders where the last order is within the period 
(t — h, t] of nonintervention before t. Hence, from time t and until time (t k + h) A {t\ + mh), 
we cannot intervene on the diffusion system and no pending order will be executed during 
this time period. This is mathematically formalized by relation (|3.7p . 

(2) Pt{k) represents the set of k pending orders where the last order is out of the period of 
nonintervention before t. Hence, at time t, one has two possible decisions : either one lets 
continue the system or one immediately intervene. In this latter case, this order adds to 
the previous ones. The mathematical formalization of these two choices is translated into 
relation (pT8l) . 

In the next sections, we show how one can exploit these dynamic programming relations 
in order to characterize analytically the value functions by means of partial differential 
equations, and then to provide an algorithm for computing the value functions. 

4 PDE system viscosity characterization 

For k = 1, . . . , m, let us introduce the subspace Q k of [0, T] k : 

@k = {t (k) = (ti)i<i< k G[0,T] fe :t k -h <mh, U - > h, i = 2,...,fc}. 

We shall write, by misuse of notation, p = (ti,ei)i<i< k = (tW,eW), for any = 
{U)i<i<k € 9fc, e( fc ) = (ej)i<j<fc € E k . By convention, we set 0^ = E k = for k = 
0. Notice that for all t € [0, T], and p = (t^ k \e^) G @ k x E k , k = 0, . . . , m, we have 

P eP t (k) ^ t€T p (k), 

where T p (k) is the time domain in [0,T] defined by : 

T„(fc) = [t k ,h+mh)n[t k ,T]. 

By convention, we set T p (k) = [0, T] for k = 0. We can then rewrite the domain T> k of the 
value function v k in terms of union of time-space domains : 

V k = {(t,x,p) : (t,x) GT p (fc) xR d , p £ 9 k x E k }. 

Therefore, the determination of the value function v k , k = 0, . . . ,m, is equivalent to the 
determination of the function v k (., .,p) on T p (k) x M. d for all p € @ k x E k . The main goal 
of this paper is to provide an analytic characterization of these functions by means of the 
dynamic programming principle stated in the previous section. 
For k = 1, . . . , m, we denote 

®T = {t {k) = (i l )i< 4 < fc G e fc : h + mh < T}, Q k (m) = Q k \ @f , 

and we define the "m-interior" of T> k by : 

= {(t,x,p)£V k :pe&fxE k }. 
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For k = 0, we set = [0, T) x R d . For p = (t i} ei)i<i< k G 6™ x we partition the time 
domain T p (k) into T p (fc) = T*(fc) U Tjj(ife) where 

Tj(fc) = |t G Tp(fc) :i<i fc + /i} = [t fc , (t fc + /i) A (h + mh)) 
T 2 p {k) = {t G Tp(fc) :t>t k + h} = [t k + h,ti+mh), 

with the convention that [s, t) = if s > t. We then partition 2?jj™ into £>™ = 2\ ,m U 2^?' m 
where 

Vl' m = {(t,x,p)€V? ■ t G Tp(fc) j 

Notice that for = 1, .. . ,m, and any p G 0™ x Tp(fc) is never empty. In particular, 
,pi,m 0^ ^ _ m ^ an( ^ an y p _ ^ ej)i<j< m G m x E™, we have t m + h >t\ + mh, 
and so Tj(m) = 0. Hence, V%T = and = Z>#" . 

The PDE system to our control problem is formally derived by sending 6 to t < t\ + mh 
into dynamic programming relations f|3.T|) - (|3.8p . This provides equations for the value 
functions v k on which take the following nonstandard form, and are divided into : 

- ^-(t,x,p) - Cv k (t,x,p) - f(x) = on Vl' m , k = l,...,m, (4.1) 



on V 2 k ' m , fc = 0,...,m-l, (4.2) 

with the convention that V^ m = T>q = [0, T) x R d . 

As usual, the value functions need not be smooth, and even not known to be continuous 
a priori, and we shall work with the notion of (discontinuous) viscosity solutions (see [7] or 
[9] for classical references on the subject), which we adapt in our context as follows. For 
a locally bounded function w k on *D™, we denote w k (resp. w k ) its lower semicontinuous 
(resp. upper-semicontinuous) envelope, i.e. 

w k (t,x,p) = liminf w k (t' , x',p'), 

(t',x',p')^(t,x,p) 

Wk(t,x,p) = limsup w k (t', x',p'), (t, x,p) G £>™, k = 0, . . . ,m. 

(t',x',p')->(t,x,p) 

Definition 4.1 We say that a family of locally bounded functions w k on T>™, k = 0, . . . ,m, 
is a viscosity supersolution (resp. subsolution) of (|4.ip - (|4,2p on T>™, k = 0, .. . , m, if : 
(i) for all k = l,...,m, (ioj^OjPo) € T> k ' m , and (p G C 2 (V^ m ), which realizes a local 
minimum of w k — ip (resp. maximum ofvJk — ip), we have 

-^r(t ,x ,po) - Cip{t ,x ) - f(x ) > (resp. < 0). 



n { ~ ~^^ x ' p ^ ~ £ v k(ti x >P) ~ f( x ) > 

v k (t,x,p) - supv k+1 (t,x,pU (t,e))\ 

«=e TP. > 
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(ii) for all k = 0, . . . , m — 1, (to,xo,Po) £ ^V" 1 ; an d ¥ G C 2 (T> k ' m ), which realizes a local 
minimum ofw k — ip (resp. maximum ofvJk~ f), we have 

dip 

miri{ - —(t ,x ,p ) - Cip(t ,x ,p ) - f(x ) , 



dt 

Wk{to,x ,p ) - swp w k+1 {t ,x ,p U (t ,e))} > 



(resp. 



dip 

min{ - — (io,xo,Po) - Cip(t , x ,p ) - f(x ) , 

Wk~(t ,x ,Po) - supwkTi(to,xo,Po U (to, e))} < 0) 

We say i/jai a family of locally bounded functions w k on T>™, k = 0,...,m, is a viscosity 
solution of (|4.ip - (|4.2p i/ i£ is a viscosity supersolution and subsolution of (|4.ip - (|4.2p . 

We then state the viscosity property of the value functions to our control problem. 
Proposition 4.1 (Viscosity property) 

The family of value functions v k , k = 0, . . . , m, is a viscosity solution to (|4,1|) - (|4.2|) . More- 
over, for all k = 0, . . . , m — 1, (t, x,p) G T>™, p = (ij, ej)i<j<fc with t = + /i, we /iaue : 

Vk(t,x,p) > sup %i (M,pU (*,e)). (4.3) 

In order to have a complete characterization of the value functions, and so of our control 
problem, we need to determine the suitable boundary conditions. These concern for k = 
1, . . . , m the time-boundary of T>™, i.e. the points (t\ +mh, x,p) for x G M. d , p = (ij, e{)i<i< k 
G 0™ x E k , and also the complement set of T>V} in P^. For a locally bounded function w k 
on X>j™, A; = 1, . . . , m, we denote 

Wk(ti + mh, x,p) = limsup w k (t,x',p'), 

(t,x ,p) — > (ti -\-mh,x,p) 

(t,x', P ') e E>jf 

w k (ti+mh,x,p) = liminf v k (t,x',p'), x G K d , p = (ij, ej)i<j<fc G 0™, 

(t, X , 33 ) — > (ti + m/i , a;, p) 

and if these two limits are equal, we set 

Wk((ti + mh)~,x,p) = Wk(t\ + mh,x,p) = w k (t\ + mh,x,p). 
We also denote for a locally bounded function wq on [0, T) x R rf : 

Wo(T,x) = limsup wo(t,x'), wo(T,x) = liminf wo(t,x'), x G M. d , 

t/T,x'-*x t/T,x'->x 

and if these two limits are equal, we set wq{T~ ,x) = wq(T, x) = wq(T, x). The complement 
set of T>™ in V k is 

V k {m) := V k \V% = {(t,x,p)£V k : p G Q k {m) x E k }. 
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Proposition 4.2 (Boundary data) 

(i) For k = 1, . . . , m, p = (U, ei)\<i< k £ 0™ x E k , x € R d , v k ((ti +mh)~ , x,p) exists and : 

v k ((tx+mh)~,x,p) = c(x,ei) + v k _ 1 (t 1 +mh, F(x,e 1 ),p-). (4.4) 

(ii) For k = 1, . . . , m, we have : 



v k (t,x,p) 



E 



f(Xl<*<»)ds + g(X^ ) , (t,x,p) € V k (m). 



(4.5) 



We can now state the unique PDE characterization result for our control delay problem. 

Theorem 4.1 The family of value functions v k , k = 0,...,m, is the unique viscosity 
solution to (14,ip - (|4.2p . which satisfy (14, 3ft . the boundary data (|4.4p -( j4~5l) . and the linear 
growth condition (|2.8|) . Moreover, v k is continuous on and on T> k {m), k = 0, . . . ,m. 

Remark 4.1 (Case m = 1) 

In the particular case where the execution delay is equal to the intervention lag, i.e. m = 
1, we have two value functions vq and v±, and the system (|4.1[) - (|4.2p may be significantly 
simplified. Actually, from the linear PDE (|4.1h and the boundary data (14. 4p for k = m = 
1, we have the Feynman-Kac representation : 



vi(t,x, (ti,ei)) 



E 



ti+h 



f(X^)ds + c{Xl% e) + voih + h, r(Xl%l e)) ,(4.6) 



t,x,0 



for all (t x , ex) G [0, T - h] X E, (t, x) 6 [ti, ti + /i) x R d . By plugging (gSJ) for t = t x into 
(14. 2h for A; = 0, we obtain the variational inequality satisfied by : 







mm 



( dv 



(4.7) 



vo — sup E 



J f(X^ )ds + c(X%^e)+vo(t + hMXt':^e))]} on [0,T-h] 



together with the terminal condition for k = (see (15. ip ) : 

cT 



v (t,x) 



E 



f(Xl^)ds + g(X^°) , (t, x) G (T - fe, T] x 



(4.8) 



Therefore, in the case m = 1, and as observed in [T4], the original problem is reduced to 
a no-delay impulse control problem (|4.7p for t>o, and t>i is explicitly related to v$ by (|4.6p . 
Equations (|4.7p - (|4.8p can be solved by iterated optimal stopping problems, see the details 
in the next section in the more general case m > 1. 

Remark 4.2 In the general case m > 1, we point out the peculiarities of the PDE char- 
acterization for our control delay problem. 

1. The dynamic programming coupled system (|4.ip - (|4.2p has a nonstandard form. For 
fixed k, there is a discontinuity on the differential operator of the equation satisfied by v k 
on Z>P. Indeed, the PDE is divided into a linear equation on the subdomain T>^ m , and 
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a variational inequality with obstacle involving the value function vt+i on the subdomain 
P^' m . Moreover, the time domain T p (k) of V™ for v k (.,x,p) depends on the argument p € 
0™. With respect to usual comparison principle of nonlinear PDE, we state an uniqueness 
result for viscosity solutions satisfying in addition the inequality (|4.3p at the discontinuity 
of the differential operator. 

2. The boundary data also present some specificities. For fixed k, the condition in (|4.4|) 
concerns as usual data on the time-boundary of the domain T>™ on which the value function 
Vk satisfies a PDE. However, it involves data on the value function v^—i, which is a priori not 
known. The condition in (|4.5p for v k concerns the complement set of Dj?, and is explicitly 
known. Notice also that we do not need to specify in Theorem 14.11 the boundary data for 
vq. Actually, this will be derived in (|5.ip as a direct consequence of (|4.5|) for k = 1 and the 
PDE equation P~2|) for k = 0. 

3. The continuity property of the value functions v k on £>j™ is not at all obvious a priori from 
the very definitions of v k , and is proved actually as consequences of comparison principles 
and boundary data for the system (|4.1|) - (|4.2|) . see Proposition 16.41 The continuity of v k 
on T>k{m) is obvious from the boundary data (|4.5|) . We mention, however, that the value 
functions v k , k > 1, are not continuous in general on their whole domain T>k '■ there is 
a discontinuity at points (T,x,p) with p = (ij, ej)i<j<fc 6 0fc s.t. t\ + rah = T. Indeed, 
from the very definition of the value functions, we have for such points v k (T, x,p) = g(x) 
(and also vq{T,x) = g(x)), while from (|4.4p . we have v k (T~ , x,p) = c(x,e±) + g(x). Hence, 
v k (T,x,p) / v k (T~,x,p) once c(x,ei) ^ 0. 

The PDE characterization in Theorem 14.11 means that the value functions are in theory 
completely determined by the resolution of the PDE system (I4.ip - (|4.2|) together with the 
boundary data (|4.4|) - (|4.5p . We show in the next section how to solve this system and 
compute in practice these value functions and the associated optimal impulse controls. 



5 An algorithm to compute the value functions and the op- 
timal control 

5.1 Computation of the value functions 

We first make the following observation. Let us denote by Fq the function defined on 
[0, T] x R d by 

F (t,x) = sup«i(t,x,(t,e)). 

eGE 

From (|4.5p for k = 1, we deduce that for all e € E, 

r r T 

F (t,x) = vi(t,x,(t,e)) 



E 



f(Xl^ )ds + g(X^) , (i, s) € (T - mh, T] 



This function Fq clearly satisfies the linear PDE : 



dF 
dt 



CF -f = 0. Hence, with g2 



for k = 0, this shows that 

Vo(t,x) = F (t,x), (t,x) e (T -mh,T] xR d , 



(5.1) 
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and in particular, vq(T~,x) = Fq(T,x) = g(x). Together with the PDE (|4.2p for k = 0, 
and a standard uniqueness result for the corresponding free-boundary problem, this proves 
that vq may also be represented as the solution to the optimal stopping problem : 



v (t,x) 



sup E[F (r, (t, x) G [0, T] x R d , 



(5.2) 



where Tt,T denotes the set of stopping times r valued in [t,T]. Hence, the value function 
vo is completely determined once we can compute v\. 

We show how one can compute v k (., .,p) on T p (k) x M. d for all p G x E k , = 1, . . . , m 
and v on [0,T] x R d . 

For & = 1, . . . , m, and any n > 1, we denote : 



so that Gfc(n) is strictly included in Gfc(ra + 1) for n = 1, . . . , N — 1, and 8fc(iV) = 0&. We 
also denote for jfe = 0, and n > 1, T n (0) = (T - nh, T] n [0, T] so that T n (0) = (T — n/i, T] 
is mcrcasme; with n = 1, . . . , N - 1, and T N (0) = [0, T\. We assumed T - mh > to avoid 
trivialities so that N > m. We denote for k = 0, . . . , m, and n = m, . . . , N , 



V%(n) = V k (n)nV^ = {(t,x,p)eV k : P eQT(n)xE k }, Qf(n) = e k (n)\Q k (m) 
Vl m (n) = V k (n)nVl m = {(t,x,p)eVr(n) :teT p (k)}, i = 1,2, 



with the convention that I?o( n ) = T n (0) x R d , so that V k {n) is strictly included in Z>fc(n+1) 
for n = 1, . . . , N — 1, and V k (N) = V k . We shall compute Ufe on V k (n), k = 0, . . . ,m, by 
forward induction on n = m, . . . , N and backward induction on A;. 

► Initialization phase : n = m. From (|4.5[) and (|5.ip . we know the values of v k on 
V k (m), k = 0, . . . , m : 



► Step n — > n + 1 for n € {m, . . . ,iV — 1}. Suppose we know the values of v k on T> k (n), 
k = 0, ... ,m. In order to determine on T> k (n + 1), fe = 0, . . . , m, it suffices to compute 
v k (.,.,p) on Tp(fc) x ]R d for all p G &f(n + l) x E k , k = 1, . . . ,m, and u on T n+1 (0) x M. d . 
We shall argue by backward induction on k = m, . . . , 0. 

• Let k = m, and take some arbitrary p — (/^, s^K^m, 

£9>|l)xr. Recall that 
Tp(m) is always empty so that T p (m) = T p (m) = [t m , 1 1 4- mh). From (|4.4p for k = 
m, we have v m ((ti + mh)~ ,x,p) = c(x, e±) + u m -i(ti + m/i, T(x, ei),p_) for all a; G 
M d , which is known from step n since either p_ G m _i(n) x E m ~ l when m > 1, or 




iV = inf {n > 1 : T - nh < 0} 
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h + mhe TP(O) when m- 1 = 0. We then solve v rn (., .,p) on T^(m) x M d from (ETll 
for k = m, which gives : 

-ti+mh 



v m (t,x,p) 



E 



f(Xrnds + c(Xl [Tmh , ei 



+ u m _i(ti +m/i,r(X^_ ( : mft ,ei),p_ 
We have then computed the value of v m (., .,p) on T p (m) x 

From k + 1 — > k for /c = m — 1, . . . , 1. (This step is empty when m = 1). Suppose 
we know the values of v k+1 (., .,p) on T p (k + 1) x R d for all p G 6^ fl (n + 1) x E fe+1 . 
Take now some arbitrary p = (ij, ej)i<j<fc G ©^(^ + 1) x We shall compute 
Vk(., -,p) successively on T p (k) x M. d (if it is not empty) and then on T p (k) x M. d , and 
we distinguish the two cases : 

(i) Tp(k) = 0. This means t k + h > t\ + m/i, and so T p (k) = T p (k) = [t k ,t\ + mh). 
We then compute v k (., -,p) on T p (/s) x R d as above for k = m : 



v k (t,x,p) 



E 



t\+mh 



fiXl^ds + ciXlf^e,) 
+ v k -i(h +m/i,r(X*^ fc ,ei),p_ 



where the r.h.s. is known from step n since either p_ € 0fc_i(n) x E k 1 when fc > 1, 
or ti + m/i G T n (0) when k - 1 = 0. 

(ii) T£(fc) ^ 0. This means + h < h + m/i, and so T^(fe) = [t fc ,t fc + /i), Tp(fc) = 
+ h,t\ + mh). For all (t,x) G T^(fe) x M d , and e e E, we have p' = pU (i,e) G 
@fcVi( n + l) x -E fc+1 , and (t, x) G T p /(A;+1) x R d . Hence, from the induction hypothesis 
at order k + 1, we know the value of the function : 

F k>p (t,x) 

We also know from step n the value of the function : 

Gk, P (x) = c(x,ei) +Ufc_i(*i +mh,T(x,ei),p-), x G M d . 

Then, from the PDE (|4.2|) and the terminal condition (|4.4|) at A;, we compute Ufc(., -,p) 
on Tp(/c) x ]R rf as the solution to an optimal stopping problem with obstacle Ff. p and 
terminal condition G k ,p ■ 



swpv k +i(t,x,pU (t,e)), (t,x) G Tp(fc) x 



v k (t,x,p) 



sup E[F feiP (r,X^- u )l 

T<ti+mh 

+ G K p{Xlf mh )l T=tl+mh ], (t, x) G T 2 p (k) x M d . 



In particular, by continuity of v k (., -,p) on T p (k), we know the value of lim 4 /t k +h v k {t, x,p) 
= Vk{tk + h,p). We then compute v k (.,.,p) on Tp(fc) x R d from (|4.ip : 



v k (t,x,p) 



E 



We have then computed the value of v k (., -,p) on T p (k) 
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• Prom k = 1 — > k = 0. From the above item, we know the value of vx(.,.,p) on 
T p (l) x M d for all p £ 6i(n + 1) x £. Hence, we know the value of : 

F (t,a;) = aupvi(t,x,(t,e)), V (i, x) € T n+1 (0) x R d . 

eS-E 

From (|5.2p . we then compute on T n+1 (0) x M d as an optimal stopping problem 
with obstacle F\. 

We have then calculated Vk(-, -,p) on T p (k) x M. d for all p <E 0™(n + 1) x E k and 
on T n+1 (0) x R d , and step n + 1 is stated. Finally, at step n = N, the computation 
of the value functions is completed since D^N) = D^, k = 0, . . . , m. 



5.2 Description of the optimal impulse control 

In view of the above dynamic programming relations, and the general theory of optimal 
stopping (see [8]), we can describe the structure of the optimal impulse control for Vq = 
Vo(Q,Xo) in terms of the value functions. Let us define the following quantities : 

► Initialization : n = 

• given an initial pending order number k = 0, we define 



: (0) 



inf{i>0 :v (t,Xf) = sup Vl (t,Xf ,(t,e))} AT, 



e\' € arg max t;i (-/-{', X A0) , (r{\e)). 

If f[ ^ + mh > T, we stop the induction at n = 0, otherwise continue to the next 
item : 

• Pending orders number k — > k + 1 (this step is empty when m = 1) from k = 1 : 

tffi = inf{<>rf + fc : 

<*(f,Jff) = sup» i+1 ((,JCf ,(ff ,ef>) 1<(< ,U(t,e))}Ar, 

4+1 e argmax^ +1 (f^ X"* 0) , (ff , ef } ) 1<i<Jk U (f£J x , e)). 

e£E T k+l 

As long as < fj ^ + mh, increment k — > k + 1 : f^ "* — > fj^)^, until 

fco = sup {A; : < + mh} € {1, . . . , m}, 
and increment the induction on n by the following step : 
► n — > n + 1 : 

• given an initial pending orders number k = k n — 1, we define 

= inf{t>(f 1 W +m/ l )V(f(: ) +/ i ) : 

w^Ct^f ,p n -) = supu fcn (t,Xf ,P„- U(t,e))} AT, 

eG-E 
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where we set p n - = (f- n \ e ™ )2<i<k n - We denote f[ n+1 ^ = if k n > 1, and f[ n+1 ^ 
= t£ +1 if k n = 1. If f} n+1 ^ + mh > T, we stop the induction at n + 1, otherwise 
continue to the next item : 

• Pending orders number k — > k + 1 (this step is empty when m = 1) from k = k n : 

fft? = M{t>f^ + h : 

v k (t,Xf) = supv k+1 (t,Xf,p n - U (rf +1) ,ef +1) ) kn<l<k U fee))} AT 

4+ + i 1} G argmax7; fc+1 (f^+ 1) ,X" ( 'l +1) ,p n - U ej n+1) ) fcn<i<fc U (f^^e)) 

As long as < f-[ n+1 ^ + m/i, increment k — > A; + 1 : f^ n+1 ^ — > t^™"^, until 

/c n+ i = sup{/c : f^ n+1 ^ < f} n+1) + mh} G {1, ...,m}, 

and continue the induction on n : n — > n + 1 until f[ n+1 ^ + m/i > T. 

The optimal impulse control is given by the finite sequence {(fj^, e k n ' > )k n _ 1 <k<k n , n = 
0, . . . , N}, where N = inf{n > : f[ + mh > T}, and we set by convention k-± = 1. 

6 Proofs of main results 

6.1 Dynamic programming principle 

From the dynamics (|2,3p of the controlled process, we derive easily the following properties 
(recall the notations j]n^(|3J^([33])) : 

• Markov property of the pair (X a ,p(., a)) for any a G ^4, in the sense that 



E 



cp(Xg) T dx = E cp(Xg) (X^,p(e 1 ,a)) 



for any bounded measurable function <p, and stopping times 0\ < 62 a.s. 

• Causality of the control, in the sense that for any a = (Tj,£j)j>i G .4, and stopping 
time, 

a 9 G and p(8,a) G k(0,a) a.s. 

where we set a 9 = {T i+i (e, a ),Z,i+u{9, a ))i>i- 

• Pathwise uniqueness of the state process, 

x t,x, P ,a = x eXe x ^ a ,P{e^), a e Qn 

for any (t, x,p) € Pfe, = 0, . . . , m, a G -4t, p , and 6* G 7^ the set of stopping times valued 
in [t,T\. 

From the above properties, we deduce by usual arguments the inequality (|3.6j) of the 
dynamic programming principle, which can be formulated equivalently in 
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Proposition 6.1 For all k = 0, . . . ,m, (t,x,p) G T> k , we have 

v h (t,x,p) < sup inf E[ f f{X?*»)ds + £ <X^ & 



t<n+mh<0 



+ v k ( dta) (9,Xl' x ' p ' a ,p(d,a)) 



Proof. Fix (t,x,p) G T> k , k = 0, ...,m, and take arbitrary a G At, p , G %.T- From 
the definitions of the performance criterion and the value functions, the law of iterated 
conditional expectations, Markov property, pathwise uniqueness, and causality features of 
our model, we get the successive relations 



J k (t,x,p,a) 



E 



+ E 



E 



t<Ti+mh<9 



9<n+mh<T 



< E 



t<Ti+mh<9 

+ Jk{e ) a)^,X t e ,x,p,a ,p(e,a),a 6 ) 

6 'f(xi>**nds+ £ c(x^_,eo 

<<Ti+m/x<6) 



+ V k (0, a )(O, X 6 ,P(Q,<*)) 

Since and a are arbitrary, we obtain the required inequality. □ 

As usual, the inequality (13. 5D of the dynamic programming principle requires in addi- 
tion to the Markov, causality and pathwise uniqueness properties, a measurable selection 
theorem. This inequality can be formulated equivalently in 

Proposition 6.2 For all k = 0, . . . , m, (t,x,p) G V k , we have 

v k (t,x,p) > sup supEff/(I^>+ Yl < X t X +mh)->& 



t<Ti+mh<6 



+ v k (0, a )(O,X e ' x ' p ' a ,p(e,a)) 
Proof. Fix (t,x,p) G T> k , k = 0, . . . , m, and arbitrary a G At, p , G % t T- By definition of 



the value functions, for any e > and lo G there exists a eu] £ A, 



■9(uj),p(9(ui),a(u))i 



which is 



an e-optimal control for v^iQt^y^^ at (9, Xg X ' p ' a ,p(9, a))(u). By a measurable selection 
theorem (see e.g. Chapter 7 in [6]), there exists a e G Ag iP (g. a ) s.t. a E (u) = a EjU (u) a.s., 
and so 



,X, 



t,x,p,a 



,p(0,a))-e < J k ( e ^ ) (6,X t d ' x ' p ' a ,p(9,a),a £ ) 



a.s. 



(6.1) 



Now, we define by concatenation the impulse control a consisting of the impulse control 
components of a until (including eventually) time r, and the impulse control components 
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of a £ strictly after time r. By construction, a G At sP , X t,x,p,a = X t)X ' p ' a on [t,0], k(9,a) = 
k(9, a), p(9, a) = p(9, a), and oP = a £ . Hence, similarly as in Proposition 16. 11 by using law 
of iterated conditional expectations, Markov property, pathwise uniqueness, and causality 
features of our model, we get 



J k (t,x,p,a) 



E 



t<Ti+mh<e 



+ Jk{6,a)(0,X e ' ' p ' ,p(9,a),a £ ) 



Together with (|6.1|) . this implies 



v k (t,x,p) > J k (t,x,p,a) > E[ f(X t a >**> a )d8+ Yl c ( X tn+mh)-^i 

* t<Ti+mh<8 
7-t,x,p,a 



+ v k (o >a )(6 ', X d ' x ' p ' a ,p(9 ,a)) 



□ 



From the arbitrariness of e, a, and 9, this proves the required result. 

We end this paragraph by proving Corollary 13.11 
Proof of Corollary 13.11 

(i) Fix k G {1, . . . , m}, (t,x) G [0, T] xR d , p= (t h ei)i<i< k G P/(/c) such that t\ + mh < 
T, and 9 stopping time valued in [t, (t k + h) A (ii + mh)). Then, we observe that for all a 
= (n,Ci)i>i e A,p, X*^^ = X*^ on [i,0], Ti + m/i > 9, k(9,a) = k, and p(9, a) = p a.s. 
Hence, relation (|3.T|) follows immediately from (|3.4p . 

(ii) For fe G {0, . . . , m — 1}, p = (U, ei)\<i< k G Pt(k) such that ii + < T, and 6> stopping 
time valued in [t, (t\ + mh) A (i + /i)). Let a = (r^, £j)j>i be some arbitrary element in At, p , 
and set r = r k+1 , £ = £ k+1 . Notice that (r,£) G J t . Then, we see that X 4 ' X 'P' Q = X l > xfi on 

Ti + mh > 9, k(9, a) = k, p(9, a) = p if 9 < t, and fc(0, a) = k + 1, p(9, a) =pU(r,() 
if 9 > t. We deduce from ((3HJ) that 

r /" e 

u fc (i,a;,p) > E[y /(X^°)d S + ^(0,X*' :[ '' o ,p)l e<T 

+ % i(fl,4 1,,0 1 pu(r ) 0)i T < s 

and this inequality holds for any (r, £) G Xt by arbitrariness of a. Furthermore, from (|3.6 
for all e > 0, there exists (r, £) G 2j s.t. 



« fc (t,x,p)-e < E / /(X^°)d S + t; fc (0,X^ u ,p)l e<T 



^.x.O 



+ V k+1 (9,Xl' X '°, P U(T,O)lr<0 

The two previous inequalities give the required relation 



v k (t,x,p) 



sup E 



f{Xl^)ds + v k {9,X^,p)l e< 



t,x,0 



+ v k+1 (9,Xl' x >°,pU(T,0)lT<9 
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6.2 Viscosity properties 

In this paragraph, we prove the viscosity property stated in Proposition 14. li We first state 
an auxiliary result, which can be proved similarly as in Lemma 5.1 in [TO]. For any locally 
bounded function u on T>™ +1 , k = 0, . . . , m — 1, we define the locally bounded function Tiu 
on T> k ' m by Tiu{t, x,p) = sup eGE u(t, x,pU (t, e)). 

Lemma 6.1 Let u be a locally bounded function on T>™ +1 , k = 0, . . . ,m — 1. Then, Tiu is 
upper- semicontinuous, and Tiu < Tiu. 

Proof. Fix some (t,x,p) G 'Dl' m , and let {t n ,x n ,p n ) n >i be a sequence in T) k ' m converging 
to (t,x,p) as n goes to infinity. Since u is upper-semicontinuous, and E is compact, there 
exists a sequence (e n ) n valued in E, such that 

Hu(t n ,x n ,p n ) = u(t n ,x n ,p n U {t n ,e n )), n > 1. 

The sequence (e n ) n converges, up to a subsequence, to some e G E, and so 

Tiu(t,x,p) > u(t,x,pU (t,e)) > limsupu(t n , x n ,p n U (t n , e n )) = limsupWu(t n , x n ,p n ) 

n— »oo n— *oo 

which shows that Tiu is upper-semicontinuous. 

On the other hand, fix some (t,x,p) G TD^' 111 , and let (t n ,x n ,p n ) n >i be a sequence in 
T>^' m converging to (t,x,p) s.t. Tiu(t n ,x n ,p n ) converges to Tiu(t,x,p). Then, we have 

Tiu(t,x,p) = lim Tiu(t n ,x n ,p n ) < lim sup 7^u(t n , x n ,p n ) < Tiu(t, x,p), 

n >oo n — >co 

which shows that Tiu < Tiu. □ 

Now, we prove the sub and supersolution property of the family Vk, k = 0, ...,m. 
There is no difficulty on the domain T>,' m since locally no impulse control is possible. 
Hence, in this case, the viscosity properties can be derived as for an uncontrolled state 
process, and the proof is standard from the dynamic programming principle (|3.7p . see e.g. 
[13] . Notice that since the domain Tp(k) is of the form [tk, (tk + h) A {t\ + mh)), we have 
no problem at the boundary. Indeed, this set is open at (t^ + h) A (t± + mh), which is 
the usual situation, and the closedness at tk does not introduce difficulties, as the value 
function is not defined before t^. Hence, when taking approximations of the upper and 
lower semicontinous envelopes of Vk, we only need to consider points of the domain such 



that t > tk, where the dynamic programming relation (|3.7p holds. The proof of the viscosity 
property of the value functions Vk to (|4.2p on V 2 k ,m is more subtle. Indeed, in addition to the 



specific form of equation (|4.2p . we have to carefully address the discontinuity of the PDE 
system (|4.ip - (|4.2p on the left boundary of T^(k). In the sequel, we focus on the domain 
V l' m , k = 0, . . . ,m - 1. 

Proof of the supersolution property on 2??' m . 

We first prove that for k = 0, . . . ,m — 1, (to, £o,Po) £ T? k ,m : 

Vk(t ,x ,p ) > sup vk + i (to,x ,p U (t ,e)). (6.2) 
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By definition of v k , there exists a sequence (t n , x n ,p n ) n >i € T>™ such that : 

Vk(tn,Xn,Pn) ^ Vk(t ,X ,p ) with (t n , X n , p n ) -> (f , X , p ) . (6.3) 

We setpo = (*°,e?) i<i<fc> Pn — (tf, e")i<i<fc, and we distinguish the two following cases : 



If to > t° + /i, then, for n sufficiently large, we have t n > i£ + h, i.e. p n G Pf n (k). 
Hence, from the dynamic programming principle by making an immediate impulse 
control, i.e. by applying (|3.8p to v k (t n , x n ,p n ) with = r = t n , and e 6 £, we have 

Vk(tn,X n ,p n ) > V k+ i (tni x ni p n U (t n , e)) > ffc+i (t n , x ni p n U (t n , e)). 

By sending n to infinity with (|6.3p . and since ffc+i is lower-semicontinuous, we obtain 
the required relation (|6.2p from the arbitrariness of e in E. 

• if to = ^ + we a Pply the dynamic programming principle by making an impulse 
control as soon as possible. This means that in relation (|3.5|) for v k (t n ,x n ,p n ), we 
choose a = (ri,Ci)i>i € A tn , Pn , = r k+1 = n := t n V (ij? + /i), £ fc+i = e £ E, so that : 

Wfc(t*,Zn,Pn) > E[[° n f(X?)ds+ Yl < X {n + mh)-^i) 

tn t n <Ti+mh<8 n 

+ v k+ i(e n ,xi, Pn u(e n ,e))]. 

Here X n := x tn,Xn '°. Since t n , 9 n — > io> Pn - ► Po> -^e„ ~~ > x o a.s., as n goes to infinity, 
and from estimate (|2.7p and the linear growth condition on /, c, Wfc+i, we can use the 
dominated convergence theorem to obtain : 

Vk{t ,x ,p ) > Vk+i {to,x ,p U (i ,e)), 

which implies (16. 2p from the arbitrariness of e € E. 

Finaly, in order to complete the viscosity supersolution property of v k to (|4.2p on ZV m , it 
remains to show that v k is a supersolution to : 

~^-(t,x,p) - £v k (t,x,p) - f(x) > 0, 

on T>f : ' m . This proof is standard by using the dynamic programming relation (|3.8|) with r 
= oo and Ito's formula, see [13] for the details. □ 



Proof of the subsolution property on T^f n . 

We follow arguments in [10]. Let (to, xo,Po) £ ^ ^ m and € C 1,2 (V^ m ) such that I>fc(to> £o>Po) 
= v9(t ,^o,Po) and ^ > on £^ ,m . If Vk(t , x ,p ) < Hvk+i(to, x ,Po), then the subso- 
lution inequality holds trivially. Now, if Vk(to, xo,Po) > Tiv k+ \(to, xo,Po), we argue by 
contradiction by assuming on the contrary that 

d 

r] ■= —Q^(t ,x ,p ) - £ip(t ,x ,Po) - f(x ) > 0. 
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We set po = (t® , e°)i<j<fc. By continuity of ip and its derivatives, there exists some 5 > 
with t + 5 < (<5 + m/i) A T such that : 

-^-C<p-f > |, on (( to -5, t + xB(i 0l <5) x%,i))n^ m . (6.4) 
From the definition of v k , there exists a sequence (t n , x n ,p n ) n >± 

B(p ,£)) n T>l ,m such that (t n ,x n ,p n ) -> (i ,^o,Po) and v k (t n ,x n ,p n ) -> Vk(t ,x ,p ) as 
n — ► oo. By continuity of <^ we also have that 7 n := Vk(t n , x n ,p n ) — ip(t n ,x n ,p n ) converges 
to as n — > oo. We set p n = (i™, e")i<j<fc. From the dynamic programming principle (|3.8p . 
for each n > 1, there exists a control (r n ,£ n ) £ 2i n such that 



Vk(tm X n , Vr, 



fS n < E 



/0X?)da + Ufc (^,X n n ,p n )l en<rn 



(6.5) 



Here X n := X*^™' , we choose n = $ n A (t„ + <f„), with tf n = inf{s > t n : X? £ B(x n , §)}, 
and (5 n ) n is a strictly positive sequence such that 

5 n — > 0, — ► 0, as n — > oo. 
On the other hand, from Lemma 16.11 we have 



Hv k+ i(t ,x ,p ) < Hv k+1 (t ,x ,p ) < v k (t ,x ,Po) < <p(to,x ,p ). 



Hence, since Tiv^+i is u.s.c. and ip is continuous, the inequality Tivk+i < <-p holds in a 
neighborhood of (to,xo,po), and so for sufficiently large n, we get : 

v k+1 (6 n ,X$ n ,p n U (T n ,£ re ))l T „<(} n < ip(e n ,Xg n ,p n )l Tn <e n a.s. 

Together with (|6.5p . this yields : 



ip(t n ,x n ,p n ) +7„--5„ < E 



f{X2)ds + tp{e n ,X% n ,p n 



By applying Ito's formula to ip(s,X™,p n ) between s = t n and s = n , and dividing by 8 n , 
we then get : 



5n 4 5 n 



< _2 E 




~ 2 





, (6.6) 



from (|6.4p . Now, from the growth linear condition on b, a, Burkholder-Davis-Gundy in- 
equality and GronwalFs lemma, we have the standard estimate : E[sup 



[ se[t n ,t n +5 n ] \ X s x n\ 2 ] 

— ► 0, so that by Chebichev inequality, Pk9 n < t n +8 n ] — > 0, as n goes to infinity, and therefore 
by definition of 8 n : 



1 > E 



'n "n 



> P[tf„ >t n + 5 r , 



1, as oo. 



By sending n to infinity into (|6.6p . we obtain the required contradiction : 



□ 
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6.3 Sequential comparison results 



In this paragraph, we prove sequential comparison results. We consider the sets Ofc(n), 
T n (0), T>k(n), V™(n), V^ m (n), introduced in Section[S]for k = 0, . . . , m, and n = m, . . . , M, 
and we define sequential viscosity solutions as follows. 

Definition 6.1 Let n € {m + 1, . . . , iV}. We say that a family of locally bounded functions 
Wk on T>™(n), k = 0, . . . ,m, is a viscosity supersolution (resp. subsolution) of (|4.ip - (|4.2p 
at step n if : 

(i) for all k = l,...,m, (io>^o>Po) £ ^fc' m ( n ); an d V £ C 1 ' 2 (^ ) fc' m ( n )); which realizes a 
local minimum of Wk — ip (resp. maximum ofWk — (p), we have 



We say that a family of locally bounded functions Wk on D™(n), k = 0, . . . , m, is a viscosity 
solution o/ ()4.ip - ()4.2p at step n if it is a viscosity supersolution and subsolution o/ ([4.ip -( j4.2p 
at step n. 

We then prove the following comparison principle at step n. 

Proposition 6.3 Let n 6 {m + 1, . . . , N}. Let Uk (resp. w^), k = 0, . . . , m, be a family of 
viscosity subsolution (resp. supersolution) of (|4.ip - (|4.2p at step n satisfying growth condition 
(|2.8p . Suppose also that Wk satisfies (|4.3p . If and are such that for all x 6 M d 



u (T,x) < wq(T,x). 
Then, uj: < on V™{n), k = 0, . . . , m. 

Remark 6.1 We recall some basic definitions and properties in viscosity solutions theory, 
which shall be used in the proof of the above proposition. Consider the general PDE 




(resp. 



mm{ - -T^{t ,x ,p ) 
Wk(to,x ,Po) ■ 



supw k+1 (t ,x ,p U (*o,e))} < 0). 



Cip{t ,x ,p ) - f(x ) , 



u k (ti + mh,x,p) < w k (ti +mh,x,p), p = (U, ei)i<i< k G @f(n) x E k , k > 1, 




D x w, D^w) 



on [t , ti) x O 



(6.7) 
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where io < ^lj an d O is an open set in R d . There is an equivalent definition of viscosity 
solutions to (|6.7p in terms of semi-jets J 2,+ w(t, x) and J 2 ~w(t,x) associated respectively 
to an upper-semicontinuous (u.s.c.) and lower-semicontinuous (l.s.c.) function w (see [7] 
or [9] for the definition of semi-jets) : an u.s.c. (resp. l.s.c.) function w is a viscosity 
subsolution (resp. supersolution) to (|6.7p if and only if for all (t,x) G [io>*i) x O, 

F(t,x,w(t,x),r,q,A) < (resp. >) 0, V(r,q,A) G J 2,+ w(t,x) ( resp. J 2 '~w(t, x)). 

For r] > 0, we say that is a viscosity 77-strict supersolution to (|6.7p . if JW*' is a viscosity 
supersolution to 

F{t,x,w\—,D x w\D 2 xW v) > v , on [to,ti)x0. 

in the sense that it is a viscosity supersolution to F(t,x,w v , — — -,IW,£>V) - 7] = 0, 
on [io,ii) x C 

As usual when dealing with variational inequalities, we begin the proof of the comparison 
principle by showing the existence of viscosity 77-strict supersolutions for equation (14.11) - 

L2D. 



Lemma 6.2 Let w k , k = 0, ... ,m, be a family of viscosity supersolutions of (|4.1|) - (|4.2|) 
satisfying (|4.3j) . Then, for any r/ > 0, £/tere exists a family of viscosity r\-strict supersolutions 
uP k of suc/t t/iai for k = 0,...,m : 

w k (t,x,p)+r ] C 1 \x\ 2 < w v k (t,x,p) < w k (t,x,p)+rjC 2 (l + \x\ 2 ), (t,x,p) G ^H 6 - 8 ) 

/or some positive constants C\, C 2 independent on i]. Moreover, for k = 0, ...,m — 1, 
(t,x,p) G £>£\ jj = (ti, ei)i<i<k with t = t k + h, we have : 

w v k (t,x,p) > supu^ +1 (i,a;,pU (i,e)) + ??. (6.9) 

Proof. For 77 > 0, consider the functions : 

w v k (t,x,p) = w k (t,x,p) + r](p 1)k (t) + r)(f> 2 (t,x), 4>i, k (t) = [(T - i) + (m - fe)] , 



2 (M) = ^-*)(l + |x| 2 ) 



1 

— < 

2 

with L a positive constant to be determined later. It is clear that w k satisfies f)6.8f) with C\ 
= 1/2 and C 2 = T + m + e LT /2. Moreover, we easily show that w k + V&ik is a viscosity 
supersolution to 

7^ ; M^fc + #i,fc) - f > V- (6-10) 

This is derived from the fact that — J £</ii t = 1, and w k is a viscosity supersolution 

at 

to - £u) fc - / > 0. We now show that (j) 2 is a supersolution to 

at 

- d -^--L4> 2 > 0. (6.11) 
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This is done by calculating this quantity explicitely. Indeed, we have 

^(t, x) = -\e L ^\\ + \x\\ CMt, x) = e^Cr-t) ( b{x)mX + tr ^ a a'{x))) . 



Since b and a are of linear growth, we thus obtain : 



-(1 + |x| 2 ) - (7(1 + |x| + \x\ 2 ) 



for some constant C independent of t, x. Therefore, by taking L sufficiently large, we 
get the required inequality (|6.1ip . which shows together with (|6.10p that w k is a viscosity 
supersolution to 



at 

Moreover, since 



^■-Cwl-f > v (6-12) 



w k (t,x,p) - sup w k+1 {t,x,p U (t,e)) > 0, 



we immediately get 



w l(^ x ^P) ~ sri Pw v k+1 {t,x,pU(t,e)) 

eeE 

= Wk(t, x,p) + r](j) hk (t) - sup w k+ i (t, x,pU (t, e)) - r]<t) 1>k+ i(t) 

> r}4) 1:k (t) - r}<t>x,k+i{t) > V- 

Together with (|6.12p . this proves the required viscosity 77-strict supersolution property for 
uP k to □ 

The main step in the proof of Proposition 16.31 consists in the comparison principle for 
77-strict supersolutions. Notice from (|6.8|) that once w k satisfies a linear growth condition, 
then w2 satisfies the quadratic growth lower-bound condition : 

r,d\x\ 2 -C 2 < w v k (t,x,p), (t,x,p)GV k n , (6.13) 

for some positive constants C\, C%. 

Lemma 6.3 Let n G {m + 1, . . . , N} and rj > 0. Let u k (resp. w k ), k = 0, . . . , m, be a 
family of viscosity subsolution (resp. rj-strict supersolution) of (|4.ip - (|4.2p at step n, with u k 
satisfying the linear growth condition (|2.8p and w k satisfying the quadratic growth condition 
([535]) . Suppose that for all x G R d , 

Uk(ti+mh,x,p) < w k (h + mh,x,p), p = (t i} ei)i<i< k G &T(n) x E k , k > 1,(6.14) 
«o(T,x) < m(T,x). (6.15) 
w k (t k + h,x,ir) > sup wfc+i (tk + h,x,pU (t k + h,e)) +rj, (6.16) 

e&E 

P = (U, ei)i<i< k G 0fc*(n) x E k , k < m - 1. 
Then, uk~ < w k on X>j™(n), k = 0, . . . , m. 
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Proof. From the linear growth of u k , and from the quadratic growth lower-bound of Wk, 
we have 

Uk(t,x,p) - w k (t,x,p) < Ci (1 + \x\) - C 2 \x\ 2 , k = 0,...,m, (t,x,p) 6 V™(ri), 

for some positive constants C±, Ci- Thus, for all k, the supremum of the u.s.c function 
Uk — Wk is attained on a compact set that only depends on C\ and C2. Hence, one can find 
ko € {0, . . . , m}, (to,Xo,po) € T^k'S 71 ) such that : 

M := sup [uk~(t,x,p) - w k (t,x,p)] 

fe e {0, . . . , m} 

(t,x, P ) 6 £>£*(") 

= n^"(to,rco,Po) - ^fc (to,rg ,po), (6.17) 

and we have to show that M < 0. We set po = e° )i<i<fc , and we distinguish the five 
possible cases concerning (ko, to> £o>Po) : 

• Case ^ : fco 7^ 0, to = + 

• Case 2 ; k = 0, t = T. 

• Case 5 : k ^ 0, t G T^(fc )- 

• Case ^ ; A; = 0, i„ G [0, T) or fc € {1, . . . , m - 1}, t € T 2 po {k ), t + t° ko + h. 

• Case 5 : ko £ {1, . . . , m — 1}, to — tk + ^- 

► Cases 1 and 2 : these two cases imply directly from (|6.14p (resp. (|6.15p ) that M < 0. 

► Cases 3 and 4 '■ we focus only on case 4, as case 3 involves similar (and simpler) 
arguments. We follow general viscosity solution technique based on the Ishii technique and 
work towards a contradiction. To this end, let us consider the following function : 

$ £ (t,t',x,x',p,p') := u^(t,x,p) - w ko (t',x',p') - tp £ (t,t',x,x',p,p'), 

with 



tp £ (t,t',x,x',p,p') = - t | 2 + b — Pol 2 ] + j\x - x \ 

1 



+— [[t - t'\ 2 + \x- x'\ 2 + [p - p'\ 2 ] . 

By the positiveness of the function ip £ , we notice that (to,xo,po) is a strict maximizer of 
(t,x,p) — > § £ (t,t,x,x,p,p). Hence, by Proposition 3.7 in [7], there exists a sequence of 
maximizers (t E , t' E , x £ , x' E ,p E ,p' E ) of $ e such that : 

(t £ ,t' £ ,x £ ,x' £ ,p £ ,p' £ ) -» (t ,to,^o,^o,Po,Po), (6.18) 
uj^(t £ ,x £ ,p £ ) - w ko (t' £ ,x' £ ,p' £ ) -> nr^(t ,xo,p ) - Wfc (to,xo,p ), (6.19) 

i[|t e -4| 2 + |x £ -x^| 2 + |p e -p;| 2 ] -> as e^0. (6.20) 
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By applying Theorem 3.2 in [7] to the sequence of maximizers (t £ , t' £ , x £ , x' £ ,p £ ,p' £ ) of 
we get the existence of two symmetric matrices A £ , A' £ such that : 

(r £ ,q £ ,A £ ) e J 2,+ uj^(t £ ,x £ ,p £ ) (6.21) 
(r' £ ,q' £ ,A' £ ) el 2 '~w ko (t' £ ,x' £ ,p' £ ), (6.22) 



where 



dib 1 
r £ = -^(t £ ,t' £ ,x £ ,x' £ ,p £ ,p' £ ) = -{t £ -t' £ ) + {t £ -t ), (6.23) 

^{t £ ,t' £ ,x £ ,x' £ ,p £ ,p' £ ) = -{t £ -Q (6.24) 



£ 



and 



with 



q £ = ^ 1 {t £ ,t' £ ,x £ ,x' £ ,p £ ,p' £ ) = -{x £ - x' £ ) + \x £ - x | 2 (x £ - x ) , (6.25) 
ax e 

cM e 1 

<?e = --Q-j(te,t' £ ,X £ ,x' £ ,p £ ,p' £ ) = -(X £ -X £ ), (6.26) 



t -4 ft" t ' 



Q(x) = 2x ® x + |x| 2 /rf, 



Id the identity matrix of dimension dx d, and for x = {xi)\<i<d G x <8) x is the tensorial 
product defined by x ® x = (a^jOi je{i d} 2 - Here, to alleviate notations, and since there 
is no derivatives with respect to the variable p in the PDE, the semi-jets are defined with 
respect to the variables (t,x), and we omitted the terms corresponding to the derivatives 
of ip £ with respect to p. We set p £ = (t? , ef)i<j< feo , and p' £ = (t'f, e- e )i<j< fco . Prom (|6.18[) . 
we deduce that for e small enoue h, i B e T2 q (/c ) and t £ / t| Q + h. From (16311 - fT22j) . and 
the formulation of viscosity subsolution of u ko to (|4.2j) and 77-strict viscosity supersolution 
of Wfc to (|4.2p by means of semi-jets, we have for all e small enough : 



min |-r e - b(x £ )q £ - ^tr (W^Ar) - f(x £ ), 



u ko (t £ ,x £ ,p £ ) - supu ko+1 (t £ ,x £ ,p £ U (t £ ,e)) } < 0, (6.28) 
min |-/ 6 - b(x' £ )q' £ - {aa' (x' £ )A' £ ) - f(x' £ ), 
Wk (t' £ ,x' £ ,p £ ) - sup w ko+1 (t' £ ,x' £ ,p £ U (t' e ,e)) } > r). (6.29) 

ee£ J 

We then distinguish the following two possibilities in (|6.28[) : 
• (i) for all e small enough, 

Uk^(t £ ,x £ ,p £ ) - supu ka+1 (t £ ,x £ ,p £ U (t e ,e)) < 0. 
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Then, for all e small enough, there exists e £ £ E such that : 

U ko (t £ ,X £ ,p £ ) < U ko+1 (t £ ,X £ ,p £ U (t £ ,e e )) + -. 

Moreover, by (|6.29p . we have 

w ko (t' £ ,x' £ ,p' £ ) > w ko+ i {t' £ ,x' £ ,p' £ U (t' £ ,e £ )) +rj. 
Combining the two above inequalities, we deduce that for all e small enough, 

Uk^{t £ ,x £ ,p £ ) - w ko (t' £ ,x' £ ,p' £ ) 

lit *n 

< u k -^l{t £ ,x £ ,p £ U (t e ,e e )) - w ko+1 (t' £ ,x' £ ,p' £ U (t' £ ,e £ )) - -. 

Since E is compact, there exists some e G E s.t. e £ — > e up to a subsequence. From 
(|6.18j) - (j6.19|) . and since tZ^", — w>fc are u.s.c, we obtain by sending e to zero : 

Uk^(to,x ,po) - w ko (t ,x ,po) 

Tj 

< u ko+ i(t ,x ,po U (t ,e)) - w ko+1 (t ,x ,p U (*o,e)) - -, 

which contradicts (|6.17p . 
(ii) for all e small enough, 

-r e - b(x £ )q £ - -tr (aa'^)^) - /(x e ) < 0. 



Combining with (|6.29p . we then get 

V < r £ -r £ + b(x £ )q £ - b(x' £ )q £ 



+ itr (aa'(x £ ),4 £ - + f(x £ ) - f(x' £ ). (6.30) 

We now analyze the convergence of the r.h.s. of (I6.30P as e goes to zero. First, we see 
from (|6.18p and (|6.23j) - (j6.24j) that r £ — r' £ converge to zero. We also immediately see 
from the continuity of / and (|6.18p that f(x £ ) — f{x' £ ) converge to zero. It is also clear 
from the Lipschitz property of b, ([fTTH]) . <K20\\ . and (l6T25]) - (p^6l ) that b(x £ )q £ -b(x' £ )q' £ 
converge to zero. Finally, from (|6.27p . we have 

3 

tr (aa'(x £ )A £ - aa'(x' £ )A' £ ) < -tr ((a(x £ ) - a(x' £ ))(a(x £ ) - a{x' £ ))') 

- tr (aa'(x £ )Q(x £ - x )) , 

and the r.h.s. of the above inequality converges to zero from the Lipschitz property 
of <7, (|6.18p and (|6.20p . Therefore, by sending g to zero into (|6.30p . we obtain the 
required contradiction : rj < 0. 

► Case 5 : We keep the same notations as in the previous case. The crucial difference is 
that uj^ and w ko may be sub and supersolution to different equations, depending on the 
position of t £ (resp. t' £ ) with respect to tf +h (resp. t' k e o + h). Actually, up to a subsequence 
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for g, we have three subcases. If t £ > t e k + h and t' £ > t' k s Q + /i for all e small enough, the 
proof of the preceding case applies. If t £ < t £ kQ + h, for all e small enough, then we have the 
viscosity subsolution (resp. supersolution) property of uj^ (resp. w ko ) to the same linear 

PDE : — — — Cv k — / = 0, at (t £ , x £ ,p £ ) (resp. (t' £ , x £ ,p' £ )), and we conclude as in Case 3. 

Finally, if t £ > i| Q + /i and t' £ < t' k e o + h for all e small enough, then the viscosity subsolution 
property of u ko to (|4.2p at (i e ,x e ,p e ), and the viscosity ^-strict supersolution property of 
w ko to (|4.ip at lead to : 

-r' £ -b(x' £ ) q ' £ -hr(aa'(x' £ )A' £ )-f(x' £ ) > r, (6.31) 

and the following two possibilities : 

-r £ -b{x £ )q £ -hi(aa f (x £ )A £ ) - f(x £ ) < 0, (6.32) 

or 

Uk^{te,x £ ,Pe) ~ supu ko+ i(t £ ,x £ ,p £ U (t e ,e)) < 0. (6.33) 

e£E 

The first possibility (|6.3ip . (|6.32|) is dealt with by the same arguments as in Case 4 (h)- 
The second possibility (|6.3ip . (|6.33[) does not allow to conclude directly. In fact, we use 
the additional condition (|6.16p : 

Wk (to,x ,p ) > sup wfc 0+ i (to,x ,po U (t ,e)) + rj. (6.34) 
Since Wk is lower semicontinuous, this implies by (|6.18p that for all e small enough : 

Wk (t' £ ,x' £ ,p' £ ) > Wka(to,Xo,po) - - 

7] 

> sup -Wfc 0+ i (to,x ,po U (t ,e)) + -. 

eG-B ^ 

Hence, by combining with (|6.33|) . we deduce that 

Uk^{t £ ,x £ ,p £ ) - w kQ (t' £ ,x' £1 p' £ ) + - 

< sup u ko+1 (t £: x £ ,p £ U (t e ,e)) - sup Wfco+i (to,x ,po U (t ,e)), 

eGB eG-E 

for all e small enough. From (|6.19p and Lemma 16.11 we then obtain by sending e to zero : 

Uk (to, x , po) - w ko (tp, x ,po) + - 

< sup u ko+ i(to,x ,Po U (io,e)) - supw ko+ i(t ,x ,p U (i ,e)) 

eG-E eG-E 

< sup|u fco+ i(t ,x ,Po U (to,e)) - ^fc +i(to,2;o,Po U (t ,e))\ . 

eGE L J 

This is in contradiction with (|6.17p . □ 

Finally, as usual, the comparison theorem for strict supersolutions implies comparison 
for supersolutions. 
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Proof of Proposition 16.31 

For any 7] > 0, we use Lemma 16.21 to obtain an ^-strict supersolution of (|4.ip - (|4.2D . 
which satisfies (|6.8p . so that w k (t, x,p) — > w^{t, x,p) for all (t, x,p) G V™, as r\ goes to zero. 
We then use Lemma [6731 to deduce that Uk < w^ 1 on £>™(n), = 0, . . . ,m. Thus, letting 
i] —> 0, completes the proof. □ 



6.4 Boundary data and continuity 

In this paragraph, we shall derive by induction the boundary data (|4.4|) - (|4.5|) in Proposition 
14.21 and the continuity of the value functions as byproducts of viscosity properties and 
sequential comparison principles. 

We first show relation (|4.5p . which follows easily from the definition of the value func- 
tions. 



Lemma 6.4 (i) For k = l,...,m,p€ @fe(w) x E k , (£, x) G T p (k) x 



we have 



v k {t,x,p) 



E 



f(Xl^)ds + g(X^' u ) 



(6.35) 



(ii) Relation (|6.35p also holds for k = 0, and for all (t,x) G T m (0) x M. d . In particular, 
Vo(T~,x) exists and is equal to vq(T,x) = g(x). 

Proof, (i) Fix k = 1, .. . ,m, p = (*i,e»)i<i<jfc G 6fe(m) x and (t,x) G T p (fc) xR d = 
fife, T] x By definition of @ k (m), we have tj + m/i > T, i = 1, . . . , k. Then, for all a = 

for t < s < T. 



t,x,p,a 



X 



t,x,0 



(7"i)£i)i>i £ *4t,p) we have Tj + m/i > T, so that from (12. 3p . X s 
We deduce immediately (16.35P from the definition of v k ■ 

(ii) This assertion was already stated in (|5.ip as a consequence of (16.35P for k = 1 and (14. 2p 
for = 0. □ 



The derivation of relation (14. 4p is more delicate. We first state the following result, 
which is a direct consequence of the dynamic programming principle. 

Lemma 6.5 (i) For k = 1, . . . ,m, and p = (ij, ej)i<j<£ G 0™ x E k , we have for all x G 
and t G T^/c) = [i fe , (t k + h) A (ii + m/i)), 

"(i fc +h)A(ti+mfc) 



v k (t,x,p) 



E 



f(Xj' x '°)ds + u fc (* fc + /i, X l t :^' h ,p)l tk+h<tl+mh (6.36) 



+ ( c ( X H+mh' e + "fc-i(*i + mh > V ( X t?+ m m e x ),pJ) ) l tl+mh < tk+h 



t,x,0 



(ii) For k = 1, . . . ,m, and p = (ii,ej)i<j<fc G 0™ x E k , such that t k + h < t\ + m/i, u;e 
Ziaue /or all x G and t G Tp(fc) = [i^ + h,t\ + mh), 



v k (t,x,p) > E 



ti+mh 



f(X^°)ds 



(ii+m/i)Ar 

/(X^ )d S + ^ +1 (r,X^°,pU(r,C))W +1 



v k (t,x,p) < sup E 



.37) 



m/i 



+ 



[ c ( x t?+ m h' e i) + + m/i,r(Z*;^,ei),p_))l tl+m/l < T . (6.38) 
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Proof. First, we recall from the dynamic programming principle that by making an im- 
mediate impulse control, i.e. by taking in (|3.8p . 9 = t and r = t, £ = e arbitrary in E, we 



have for all k 
t k + h, 



0. 



, m 



l,p = (U,ei)i<i< k € G fe x E k , (t,x) G T p (fe) x M d with i > 



v k {t,x,p) > swpv k+1 (t,x,pU(t,e)). 



(6.39) 



We distinguish 

t,x,0 



(i) Fix k = 1, . . . , m, p = (t h ei)i<i< fe G 6™ x and (i, x) G Tj(fc) x I 
the two following cases : 

• Case 1 : t k + h < t\ + mh. Then, for all a G At p , we have from (|2,3p . Xs' x,p ' a = X L 
for i < s < t k + h. Hence, by applying (13. 4p with 6* = tk + fo, and noting that r% + m/i > 9, 
k(9,a) = k, p{9,a) = p for any a = (rj,^) G At, p , we obtain the required relation (|6.36|) . 
i.e. 



v k (t,x,p) 



E 



t k +h 



Case 2 : t\ + m/i < + h. Then, for all a G At >p , we have from (|2.3p . X. 



t,x,p,a 



Xt^° for 



i < s < ii + m/i, and -2T^+^ = r(X t '^_ mh , ei). Hence, by applying (|3.4p with 6* = ii + m/t, 
and noting that for any a = (ji, £*) G «4.t, p , we have either k(9,a) = k — 1, p(9,a) = p_ 
if Tfc + i > ii + m/i (which always arises when ii + m/i < i/. + /i), or k{9, a) = k, p(9, a) = 
p- U (r k+ i, £ fe+ i) if T fc+ i = i fe + ft = ti + mh, we obtain 



v k (t,x,p) 



sup E 



t\+mh 



f(Xl^)ds + c(X^ mh ,e 1 



t,x,0 



+ + m/i,r(X*;^,ei),p_)l Tfc+1>il+mh 



+ u fc (ti + ?nh,F(X t t ' i x f mh ,e 1 ),p- U (ii + mh, € k+ i))l Tk+1 =ti+mh=t k +h 



Now, from (|6.39p . if t\+mh = t k +h, we have Vk{t\+mh,T{X t t '^ rnh , ei),p-Ll(ti+rnh, Ck+l)) 
< v k -i(ti + mh,T{X t t '^ mh ,ei),p-) for all ^k+i ^tj+m/i-measurable valued in E. We then 
deduce 



v k (t,x,p) 



E 



ti+mh 



f(X^°)ds + c(X^ mh , ei ) + Vk-Ah + m/i, r(X^° m/i; 



ei),P- 



which is the required relation (|6.36p . 

(ii) Fix k = 1, . . . , m, p = (ti, ej)i<j<fc G 6™ x E k , s.t. tk + h < ti + m/i, and (i, x) G 



X^' for t < s < t\ + m/i, 



T^(fe) x M d . Then, for all a G A t>p , we have from ([13]), X. 
and X^_^^ 

= r(X t'+ m/l ,ei). Let a = (Tj,&) be some arbitrary element in and set 
r = Tk+i, £, = 6fc+i- Observe that with 9 = (ti + m/i) A r, we have a.s. either k(9,a) = 
k + 1, p(9, a) = p U (r, £) if r < ii + m/i or a) = fe — 1, p(0, a) = p- if r > ti + m^i, 
or A:(6*, a) = /c, p(0, a) = p_ U (r, £) if r = t\ + m/i. Hence, by applying (|3.5p to some a = 
(rj,^j) G ^Lj iP s.t. Tfc+i > ti + m/i a.s. and with 9 = t\ + mh, we get the inequality (|6.37p . 
Furthermore, from (|3.6p . for all e > 0, there exists a = (rj,^j) G At, p s.t. by setting r = 
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Tk+i, £ = £fc+l> and with = (*i + mh) A r, 

v k {t,x,p)-e < f{Xl^)ds + v k+1 {r,X^\ P ^{r,0)^T<t^ m h 

+ c ( x t*+ m hi ei)l tl + m h<r + v k -i(ti + mh, r(X*^ /l , ei),p_)l il+m / l<r 
+ u fe (ti + mh,T(xl'^° mh ,e 1 ),p- U (t, £))l T=tl+mh . 

Now, wehmev k {ti+mh,T(Xl! L x ^ mh ,ei),p-U(t 1 +rnh,^)) < v k -x(t x +mh, r(X*^ m/l , ei),p_) 
from (|6,39p . Since (r, £) € T t , and e is arbitrary, we deduce the required relation (|6,38p . □ 

Proposition 6.4 For all k = 0, ... ,m, v k is continuous on T> r k n and T> k (m). Moreover, 
for all k = 1, . . . ,m, p = (U, ei)i<i< fc £ 6™ x E k , x e R d , 

v k ((ti + mh)~,x,p) = c(x,ei) +v k _i(ti + mh,T(x,e 1 ),p-). 

Proof. 1. We first easily see from Lemma |6.4| continuity and growth condition of /, g, 
and dominated convergence theorem, that v k is continuous on T> k (m), for all k = 0, . . . , m. 
2. We shall prove by forward induction on n = m + 1, . . . , N that (Hk)(n), k = 1, . . . , m, 
and (H0)(n) hold, where 

(Hk)(n) v k is continuous on 2?™(n), and for all p = {ti,ei)\<i< k € S^(n) x E k , 
v k ((ti + mh)~ ,x,p) = c(x, ei) + + mh,F(x, ei),p_), x 6 M d . 

(H0)(n) v is continuous on TT(0) x R d . 

► Initialization : n = m + 1. Let us prove that (Hk)(m+1), A; = l,...,m, and 
(H0)(m+1) are satisfied. 

• Take some k = 1, . . . , m, and fix some arbitrary x € M. d and p = (ij, ej)i<j<fc £ 0™(m+ 1) x 
E k . Notice that p- £ Q k -i(m) x E^" 1 so that v k -\(., is continuous on T p _ (k — 1) x M d 
from part 1. above. Here, to alleviate notations, we used the convention that T p _(k — 1) 
_ r jp m (0) if A; — 1 = 0. We distinguish two cases : 

* Case 1. T 2 p (k) = 0, i.e. h + mh < t k + h so that T p (fc) = Tj(ife) = [t fc ,ti + m/t). From 
(j6.36p . we then have for all t € T p (k) : 

v k (t,x,p) = E[ y f(Xl' x >°)ds + c(X*f^, ei ) + i^fa + m/ i ,r(X^,e 1 ),p_) 

By continuity of + mh, -,p~), ei), c(., ei), growth condition on /, c, T and 

we deduce with the dominated convergence theorem that v k {{t\ + mh)~,x,p) exists and 

v k ((ti + mh)~,x,p) = c(x,ei) +v k - 1 (t 1 + mh, F(x,e 1 ),p-). 

-k Case 2. T p (k) = [t k + h,t\ + m/i) 7^ 0, i.e. t\ + m/i > t k + h (this implies in particular 
that k < m and m > 1). From (|6.37p - (|6.38p . we first prove that 

Uk(ti + mh,x,p) 

< max [c(x, ex) + v k -i(ti + mh, x,p-), sup v k+ \{t\ + mh, x,pU (t\ + mh, e))](fi.40) 
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Indeed, consider some sequence (t e , x £ ,p £ ) £> o £ V™ converging to (t\ + mh, x,p) and such 
that lim £ ^o v k (t £ , x e ,p e ) = Vk{t\ + mh,x,p). For any e > 0, one can find, by (|6.38p . some 
(r £ ,ie) e l te s.t. 



v k {t e ,x £ ,p £ ) < E 



(tf +mh) Af s 



+ {< X qZl 4) + Vk-M + mh, ef),p e _)J l tf+m/l <f s 

where we denote p e = (if, ef)i<j<fc and p e _ = (if , ef) 2 <i<k- By setting 

G £ = c(X^°, e f)+,; fe _ 1 (tf + m/ i ,r(X^, e f),p e _), 
we rewrite the above inequality as 

f(tf +mh)Af s 

v k (t £ ,x £ ,p £ ) < E 



/(X^^ds + Ufe+1 (f £ ,4f £ '°,p £ U (f e ,6))lf e<t!+m fc 



f(Xl*> x ^)ds + G £ 
+ (« fc+ i(f e ,X^ !B ' ,0 ,p e U (f £ ,4)) - G B W <t j +mA 



.41) 



Since p_ 6 Ofc_i(m) x E , we have p e _ 6 0&_ i(m) x for e small enough. Hence, by 
continuity of v k -\ on T> k -i(m) (from part 1.), continuity of T and c, and path-continuity 
of the flow Xt ,x ' , we have 

lim G £ = G := c(x, e±) + Vk-i(ti + mh, T(x, e\),p-) a.s. (6.42) 

£— >0 

Moreover, by compactness of E, the sequence (£ 6 ) 6 converges, up to a subsequence, to some 
£ valued in E. We deduce that 

lim sup (v k+1 (f £ ,Xi £ ' Xe '°,p £ U (f e ,| £ )) - G e )l fe<if+m/ j 



< [Vk+i(h +mh,x,pU (ti +mh,£)) - G \ limsup lf e <q+mh 



< ( supt>fc+i(ii + m/t, x,p U (t\ + m/i, e)) — G J limsup lf e <tf+mft (6.43) 



From the linear growth condition on /, c, T, v k -i, v k+ \, and estimate (|2.7p . we may use 
dominated convergence theorem and send e to zero in (|6.4ip to obtain with (|6,42p - (|6.43p : 



Vk(t\ + mh,x,p) 



< E 



G+ ( supw fe+ i(ti +mh,x,pU (ti + mh,e)) - G) limsup lf s <tf+mfe 



< max 



G, sup Vfc+i(ti + m/i, x, p U (ti + mfa, e)) 



which is the required inequality (|6.40p . 
We next show that 

sup vj.-\-i (ti + mh, x,pU (t± + mh, e)) < c(x, ei) + v k _i(t\ + mh, x,p-). (6.44) 

e€E 
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Indeed, for any arbitrary eGfi, consider some sequence (t £ ,x £ ,p £ , e £ ) £> o G V™ x E converg- 
ing to (t± + mh, x,p, e) and such that lim e ^o v k+i(t £ , x £ ,p £ U (t £ , e £ )) = Vk+\(ti + mh, x,pU 
(t\ + mh, e)). For e small enough, t £ + h >t\+ mh, and so from the DPP (|6.36p . we have : 

v k+1 (t £ ,x £ ,p £ U(t £ ,e £ )) = E / f(Xl^o )ds + c ( X i W j6 |) 

+ Ufc (i? + m/i,r(X^°,ef),p £ _ U (i £ ,e £ ))] . (6.45) 

Since p- G 0fc_i(m) x E k ~ l , we have p £ _ 6 0fc_i(m) x E* -1 for e small enough. Hence, 
by continuity of on T>k(m), continuity and growth linear condition of /, V and c, and 
path-continuity of the flow Xl ,x, °, we send e to zero in (|6.45p and get by the dominated 
convergence theorem 

v k+i {p\ + m ^-; i,pU (ti + m/i, e)) 
= c{x,e\) + Vk{t\ + mh,T(x,ei),p~ U (t\ + mh,e)). (6.46) 

Moreover, from (|6.39p . we have Vk-i{t\ +mh, T(x, e\),pJ) > Vk(t\ + mh, T(x, e±),p- U (t\ + 
mh,e)) for all e £ £. Plugging into (|6,46p . this proves (|6.44j) . 
Finally, we easily see from (j6.37p that 

Vk(ti + mh, x,p) > c(x, ei) + Vk-i(t\ + mh, x,p-). (6.47) 

Indeed, consider some sequence (t e , x £ ,p £ ) £> o G V™ converging to (t\ + mh,x,p) and such 
that lim £ ^o Vk{t £ ,x £ ,p £ ) = Vk[t\ + mh,x,p). From (|6.37p . we have in particular 

v k {t £ ,x £ ,p £ ) > E / f(Xl^°)ds 
L Jt e 

+ 4) + «*-!(*! + mh, T{X\r^ h , e\),p E „) . 

By continuity and linear growth condition of Vk-\, T, c, f, and estimate (|2.7p . we get (|6.47p 
by the dominated convergence theorem, and sending e to zero in the above inequality. 

Hence, the inequalities (|6.40j) - ()6.44[) - (j6.47|) prove that Vk{{t\ + mh)~ ,x,p) exists and is 
equal to : 

Vk((ti + mh)" , x,p) = Vk{t\ + mh,x,p) = Vk(t\ + mh,x,p) (6.48) 
= c(x, ei) + v k -i(h + mh, F(x, e x ),pJ). 

We have then proved that (|6.48j) holds for all k = 1, . . . , m, p = (U, ei)\<i<k G Q^im + 
1) x E k , and x G R d . 

• We know from Proposition 14.11 that the family of value functions Vk, k = 0, . . . , m, is a 
viscosity solution to (|4.ip - (|4.2p . in particular at step n = m + 1. We also recall from Lemma 
16.41 that Uq(T,x) = vq{T,x) = g(x). Together with (|6.48p . and the comparison principle 
at step n = m + 1 in Proposition 16.31 this proves Uk < Vk on £>™ (n) for n = m+1. This 
implies the continuity of Vk on Z?fc(m + 1), k = 0, ... ,m, and so (Hk)(m-|-1), k = 1, . . . ,m, 
and (H0)(m-|-1) are stated. 
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► Step n — > n + 1 for n G {m + 1, . . . , N — 1}. We suppose that (Hk)(n), k = 1, . . . ,m, 
and (HO)(n) hold true. 

Take some k = 1, . . . , m, and fix some arbitrary x € M. d and p = (U, ei)\<i<k G (G™(n + 
1) x ¥, k . Notice that p- G @k-i( n ) x E ^. By same arguments as in step n = m + 1, using 
here, instead of part 1., continuity of on PP,(n) by the induction hypothesis of step 
n, we prove that 

Vk((ti + mh)~,x,p) = Vk(h + mh, x,p) = Vk_(h + rnh,x,p) 
= c(x, ei) + ujfc_i(ti + m/i, T(x, e\),pJ). 

We also have vq(T,x) = vq(T,x) = g(x). Therefore, from the viscosity property of Vk, 
k = 0, . . . ,m, to (|4.ip - (|4.2p at step n + 1, and the comparison principle in at step n + 1 
in Proposition 16.31 we obtain vj: < Vk on T> k n (n + 1), which implies the continuity of 
on T> k n (n + 1), = 0, . . . , m. Therefore, (Hk)(n+1), k = 1, . . . , m, and (H0)(n+1) are 
proved. 

► The proof is completed at step N by recalling that @ k (N) = G fc , T>™{N) =Vf,k = 

0, ...,m. □ 

6.5 Proof of Theorem 14.11 

In view of the results proved in paragraphs 16.21 and 16.41 it remains to prove the uniqueness 
result of Theorem 14. 1[ Let us then consider another family Wk, k = 0, . . . , m of viscosity 
solutions to (|4.ip - (|4.2p . satisfying growth condition (|2.8p . and boundary data (|4.4p - (|4.5p : 

for k = 1, . . . , m, p = (t h ei)i<i< k G 0^ x E k , x e R d , 

w k ((h + mh)~ ,x,p) = c(x,ei) + w k -i(h + mh,T(x,ei),p-). (6.49) 

and 

w k (t,x,p) = f(Xt^ )ds + g(X^ xfi )\, (t,x,p)eV k (m). (6.50) 

We shall prove by forward induction on n = m, . . . ,N that v k = Wk on T>k(n). 

► Initialization : n = m. Relations (|4.5p . (|6.50p and ()5.ip show that Vk = Wk on T>k(m), 
k = 0, . . . ,m. 

► Step n — ► n + 1. Suppose that = on T>k(n), k = 0,...,m. For any > 

1, p = (tj,ej)i<j<fc G 6™(n + 1) x we notice that p_ G &k-i( n ) x Hence 
Vk-i(t\ + m/t, T(x, ei),p_) = Wk-i{ti + m/i, r(x, ei),p_), x G and so from (|4.4p . (|6.49p . 
we have 

+m/t) _ ,x,p) = w k ((h + mh)~ ,x,p). 

We already know that uo(T - ,:r) = wq(T~,x) (= g(x)). Therefore, from the comparison 
principle at step n + 1 in Proposition 16.31 we deduce that u k = w k on £>™(n + 1), and so 
on T>k(n + 1), k = 0, . . . ,m. Finally, the proof is completed since T>k{N) = T>k- 
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